<html>
<head><meta charset="utf-8"><title>vec_append_from_within · wg-secure-code · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/index.html">wg-secure-code</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/vec_append_from_within.html">vec_append_from_within</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="168636954"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146229-wg-secure-code/topic/vec_append_from_within/near/168636954" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Shnatsel <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/vec_append_from_within.html#168636954">(Jun 20 2019 at 21:38)</a>:</h4>
<p>This "append part of vector to itself" problem has caused bugs in two crates and even a CVE in stdlib. I'm writing an RFC for a stdlib abtraction to fix this once and for all. Sneak peek:</p>
<div class="codehilite"><pre><span></span><span class="n">assert_eq</span><span class="o">!</span><span class="p">(</span><span class="n">vec</span><span class="o">!</span><span class="p">[</span><span class="mi">3</span><span class="p">,</span><span class="mi">5</span><span class="p">,</span><span class="mi">7</span><span class="p">].</span><span class="n">append_from_within</span><span class="p">((..</span><span class="mi">1</span><span class="p">)),</span><span class="w"> </span><span class="n">vec</span><span class="o">!</span><span class="p">[</span><span class="mi">3</span><span class="p">,</span><span class="mi">5</span><span class="p">,</span><span class="mi">7</span><span class="p">,</span><span class="mi">3</span><span class="p">]);</span><span class="w"></span>
<span class="n">assert_eq</span><span class="o">!</span><span class="p">(</span><span class="n">vec</span><span class="o">!</span><span class="p">[</span><span class="mi">3</span><span class="p">,</span><span class="mi">5</span><span class="p">,</span><span class="mi">7</span><span class="p">].</span><span class="n">append_from_within</span><span class="p">((</span><span class="mi">1</span><span class="p">..)),</span><span class="w"> </span><span class="n">vec</span><span class="o">!</span><span class="p">[</span><span class="mi">3</span><span class="p">,</span><span class="mi">5</span><span class="p">,</span><span class="mi">7</span><span class="p">,</span><span class="mi">5</span><span class="p">,</span><span class="mi">7</span><span class="p">]);</span><span class="w"></span>
<span class="n">assert_eq</span><span class="o">!</span><span class="p">(</span><span class="n">vec</span><span class="o">!</span><span class="p">[</span><span class="mi">3</span><span class="p">,</span><span class="mi">5</span><span class="p">,</span><span class="mi">7</span><span class="p">].</span><span class="n">append_from_within</span><span class="p">((..)),</span><span class="w"> </span><span class="n">vec</span><span class="o">!</span><span class="p">[</span><span class="mi">3</span><span class="p">,</span><span class="mi">5</span><span class="p">,</span><span class="mi">7</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span><span class="mi">5</span><span class="p">,</span><span class="mi">7</span><span class="p">]);</span><span class="w"></span>
<span class="n">vec</span><span class="o">!</span><span class="p">[</span><span class="mi">3</span><span class="p">,</span><span class="mi">5</span><span class="p">,</span><span class="mi">7</span><span class="p">].</span><span class="n">append_from_within</span><span class="p">((..</span><span class="mi">20</span><span class="p">));</span><span class="w"> </span><span class="c1">// panic!</span>
</pre></div>



<a name="168637081"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146229-wg-secure-code/topic/vec_append_from_within/near/168637081" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Shnatsel <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/vec_append_from_within.html#168637081">(Jun 20 2019 at 21:40)</a>:</h4>
<p>Do I understand correctly that if you want to read several elements from the vector, modify them in some way and append them to the vector you'd have to copy them onto the stack or a different heap location? Otherwise modification would affect the elements within the vector, right?</p>



<a name="168637585"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146229-wg-secure-code/topic/vec_append_from_within/near/168637585" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Shnatsel <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/vec_append_from_within.html#168637585">(Jun 20 2019 at 21:48)</a>:</h4>
<p>Oh yeah, important constraint: I also want to push them all at the same time</p>



<a name="168641541"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146229-wg-secure-code/topic/vec_append_from_within/near/168641541" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Tony Arcieri <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/vec_append_from_within.html#168641541">(Jun 20 2019 at 22:56)</a>:</h4>
<p>that sounds like the safest way to avoid aliasing issues to me...</p>



<a name="168641594"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146229-wg-secure-code/topic/vec_append_from_within/near/168641594" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Tony Arcieri <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/vec_append_from_within.html#168641594">(Jun 20 2019 at 22:57)</a>:</h4>
<p>so is the issue people are doing unsafe in-place modifications and running into aliasing bugs?</p>



<a name="168644725"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146229-wg-secure-code/topic/vec_append_from_within/near/168644725" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Shnatsel <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/vec_append_from_within.html#168644725">(Jun 20 2019 at 23:56)</a>:</h4>
<p>The issue is people are trying to append large chunks of the vector to itself. There is no safe way to do that, so they hand-roll unsafe code that does <code>set_len()</code> on vector and then fill it in by copying from the already initialized part. RLE implementations end up not handling the degenerate case where the offset for copying is 0 and they just iterate over the buffer reading uninitialized bytes and writing them back in their place. Stdlib dodged that (hopefully - I haven't checked), but didn't handle the overflow in length computation and ended up with a buffer overflow bug, see CVE-2018-1000810</p>



<a name="168645086"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146229-wg-secure-code/topic/vec_append_from_within/near/168645086" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Shnatsel <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/vec_append_from_within.html#168645086">(Jun 21 2019 at 00:04)</a>:</h4>
<p>The RFC is almost complete, here's what I've got so far: <a href="https://gist.github.com/Shnatsel/e3907b56921e1b6be187aad1d86efc3e" target="_blank" title="https://gist.github.com/Shnatsel/e3907b56921e1b6be187aad1d86efc3e">https://gist.github.com/Shnatsel/e3907b56921e1b6be187aad1d86efc3e</a></p>



<a name="168675056"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146229-wg-secure-code/topic/vec_append_from_within/near/168675056" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Gaynor <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/vec_append_from_within.html#168675056">(Jun 21 2019 at 12:00)</a>:</h4>
<p>FWIW your python example has slightly different semantics, since slicing a list performs a copy, so:</p>
<blockquote>
<p><code>.extend</code> method works even when passing a subslice of the same list<br>
Is mostly irrelevant since subslices aren't a thing.</p>
</blockquote>



<a name="168713019"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146229-wg-secure-code/topic/vec_append_from_within/near/168713019" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Shnatsel <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/vec_append_from_within.html#168713019">(Jun 21 2019 at 20:18)</a>:</h4>
<p>Thanks, I've amended it to make that more clear. Also replaced the C example with a memcpy.</p>



<a name="168713027"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146229-wg-secure-code/topic/vec_append_from_within/near/168713027" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Shnatsel <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/vec_append_from_within.html#168713027">(Jun 21 2019 at 20:18)</a>:</h4>
<p>RFC is now up for review: <a href="https://github.com/rust-lang/rfcs/pull/2714" target="_blank" title="https://github.com/rust-lang/rfcs/pull/2714">https://github.com/rust-lang/rfcs/pull/2714</a></p>



<a name="169412364"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/146229-wg-secure-code/topic/vec_append_from_within/near/169412364" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Tony Arcieri <a href="https://rust-lang.github.io/zulip_archive/stream/146229-wg-secure-code/topic/vec_append_from_within.html#169412364">(Jul 01 2019 at 17:07)</a>:</h4>
<p>subscribed to the issue</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>